26
Grading Evidence in Medicine Bill Cayley Jr MD MDiv UW Health Augusta Family Medicine

Grading Evidence in Medicine

Embed Size (px)

DESCRIPTION

evidence in medicine

Citation preview

  • Grading Evidence in MedicineBill Cayley Jr MD MDivUW Health Augusta Family Medicine

  • ObjectivesParticipants will be able to:Describe the practice of evidence-based medicineDiscuss the presentation and classification of evidenceDiscuss grading of evidence and integration into clinical practice

  • What isevidence-based medicine?

  • Two fundamental questionsWhat is the purpose of medicine?How do I decide what to do?

    You have to know where youre going before deciding how to get there

  • What is the purpose of medicine?Patient carePublic healthResearch

    Improving the quality of patients lives

  • What is evidence-based medicine?Evidence based medicine is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients.

    Sackett, et al. BMJ 1996;312:71-72

  • What is EBM NOT?What we have always doneCookbook medicineOnly a cost-cutting trickOnly randomized trials

    Evidence based medicine ISTracking down the best external evidence with which to answer our clinical questions

  • EBM a short historyJAMA 1992EBM: a new approachJAMA 1993 2000Users' Guides to the Medical Literature1990s 3 trendsSystematic reviewsSearch enginesKnowledge distillation and push services

  • Classification of evidence

  • How do I decide what to do?The answer from EBMuse of current best evidence

  • Evidence: systematic observation

    Meta-AnalysisRandomized Controlled TrialUncontrolled TrialCase SeriesAnecdote

  • Evidence grading1989 USPSTF5 levels of evidenceOther systems:CEBMACCAAFP (SORT)GRADEDetail, vs practicality

  • USPSTF (as of May 2007)

    GradeDefinitionSuggestions for PracticeAThe USPSTF recommends the service. There is high certainty that the net benefit is substantial.Offer or provide this service.BThe USPSTF recommends the service. There is high certainty that the net benefit is moderate or there is moderate certainty that the net benefit is moderate to substantial.Offer or provide this service.CThe USPSTF recommends against routinely providing the service. There may be considerations that support providing the service in an individual patient. There is at least moderate certainty that the net benefit is small.Offer or provide this service only if other considerations support the offering or providing the service in an individual patient.DThe USPSTF recommends against the service. There is moderate or high certainty that the service has no net benefit or that the harms outweigh the benefits.Discourage the use of this service.I StatementThe USPSTF concludes that the current evidence is insufficient to assess the balance of benefits and harms of the service. Evidence is lacking, of poor quality, or conflicting, and the balance of benefits and harms cannot be determined.

  • GRADE (a work in progress)GRADE classifies recommendations as strong or weakStrong recommendations mean that most informed patients would choose the recommended management and that clinicians can structure their interactions with patients accordinglyWeak recommendations mean that patients choices will vary according to their values and preferences, and clinicians must ensure that patients care is in keeping with their values and preferences

    Strength of recommendation determined by the balance between desirable and undesirable consequences of alternative management strategies, quality of evidence, variability in values and preferences, and resource use

  • Meta-AnalysisRandomized Controlled TrialUncontrolled TrialCase SeriesAnecdote

    More systematic observation better evidence

  • Integrating evidence & practice

  • What type of outcome measures?Surrogate markers of disease:Hb A1c, cholesterol, blood pressureStage or extent of disease:Diabetic ulcers, angiographic CAD, strokePatient-oriented outcomes:Mobility, suffering, longevityMorbidity and mortality

  • Patient or disease oriented?Disease-Oriented Outcomes. Intermediate, histopathologic, physiologic, or surrogate resultsExamples: blood sugar, blood pressure, flow rate, coronary plaque thicknessMay or may not reflect improvement in patient outcomes.Patient-Oriented Outcomes. Outcomes that matter to patients and help them live longer or better livesExamples: including reduced morbidity, reduced mortality, symptom improvement, improved quality of life, or lower cost

  • Which outcomes????Topical antibiotics for bacterial conjunctivitis may improve early and late resolution rates, but nearly all cases ultimately have complete remission.Br J Gen Pract. 55: 962-4.Digoxin for symptomatic heart failure provides no significant difference in mortality but is associated with lower rates of hospitalization and of clinical deterioration. J Card Fail. 10:155-64.Long-acting beta-2 agonists for asthma are effective in reducing symptoms but may increase mortality or exacerbations.Cochrane Database of Systematic Reviews 2008, Issue 3. Art. No.: CD006363.

  • SORT

  • When guidelines conflictQuestions of evidenceWere the clinical questions different?Were different studies considered?Were the results analyzed differently?Was the quality of evidence assessed differently?

    Questions of outcomesDid the effect estimates for important outcomes differ?Did judgments about evidence quality differ?Were health consequences weighed differently?Were economic consequences considered differently?

  • Systems applications

  • Informed decision-makingPhysicians must recognize the role of being the patients agent in helping make informed decisions to maximize benefit at reasonable costMedical students and residentsshould be educated to approach care as the patients agent in making informed decisions, rather than solely as an autonomous decision makerThe evidence for and approach to developing standards should be standardized.

  • In shortEBM is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients.

    EvidenceSystematic observation = high-quality evidence

    Patient-oriented evidencepreferable to Stage of diseasepreferable toSurrogate markers

  • For more informationAbout EBMCentre for Evidence-Based Medicine (http://www.cebm.net/)Agency for Healthcare Research and Quality (http://www.ahrq.gov/clinic/epcix.htm)

    Evidence sourcesDynaMed (www.dynamicmedical.com/)Essential Evidence Plus (www.infopoems.com/)Cochrane Library (www.cochrane.org/)Database of Abstracts of Reviews of Effectiveness (www.crd.york.ac.uk/crdweb/)FPIN (www.fpin.org/) Clinical Evidence (www.clinicalevidence.com/)

  • **Introductory question for participants: How would you define evidence-based medicine?*Two fundamental questions need to be addressed in medical practice: First, what is the purpose of medicine? Second, how do I decide what to do? Just as a person planning a cross-country trip maps out the journey based on decisions about the most desirable destination and the most desirable routing, so in medicine physicians must work with patients to decide on both overarching goals, and on the best way to reach those goals. In both travel and medicine, you have to know where youre going before deciding how to get there. *While medicine has many facets, including direct patient care, public health, and research endeavors, the common goal of all medical ventures is improving the quality of patients lives. This common underlying goal can serve as the destination to guide all medical journeys and endeavors, regardless of which facet or field of medicine is the stage for the patients medical journey. *In a much-quoted 1996 editorial, David Sackett, an early and prolific author on evidence-based medicine, defined evidence-based medicine (EBM) as the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. (A) (italics added) This characterization highlights the three important parts of evidence-based medical practice: the patient, the evidence, and careful application of generalized evidence to the individual patient.

    (A) Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. BMJ. 1996 Jan 13;312(7023):71-2. PMID: 8555924 (http://www.bmj.com/cgi/content/full/312/7023/71)*Sackett goes on to argue that EBM is not old hat or just the same old medical practice, as evidenced by the wide variation that often occurs in clinical practice patterns. Neither is it intended to be cookbook medicine as EBM specifically advocates for individualized application of evidence to patient care, not forcing patient care to conform to generalized evidence. Further, EBM is not intended primarily as a savings tool it is intended to guide practitioners to provide the best, not necessarily the cheapest, care. Lastly, EBM is not intended to be only concerned with randomized controlled trials, but with the best relevant evidence applicable to the situation in question.

    (A) Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. BMJ. 1996 Jan 13;312(7023):71-2. PMID: 8555924 (http://www.bmj.com/cgi/content/full/312/7023/71)*Evidence-based medicine was first introduced in the mainstream medical literature in a 1992 article, Evidence-based medicine: A new approach to teaching the practice of medicine, which presented EBM as a fundamentally new approach emphasizing question formulation, search and retrieval of the best available evidence, and critical appraisal of the study methods to ascertain the validity of results. (B) A subsequent series of Users' Guides to the Medical Literature presented skills for searching for, appraising, and applying various types of published evidence to medical practice. As EBM gained prominence in the late 1990s and beyond, three streams of evidence dissemination developed: 1) systematic reviews gained increasing prominence in the medical literature, 2) knowledge search engines (including internet engines such as Google, and medline interfaces such as Ovid) became standard tools for medical literature searching, and 3) knowledge distillation and push services developed as a way to compile and disseminate concise reviews of evidence on specific topics or questions (eg, ACP Journal Club, InfoPoems, etc). (B)

    (A) Evidence-Based Medicine Working Group. Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA. 1992 Nov 4;268(17):2420-5. PMID: 1404801 (http://jama.ama-assn.org/cgi/reprint/268/17/2420?ijkey=d3d27e0bf59a836b2ff7923ef06634c6304b1c75&keytype2=tf_ipsecsha)

    (B) Montori VM, Guyatt GH. Progress in evidence-based medicine. JAMA. 2008 Oct 15;300(15):1814-6. PMID: 18854545 (http://jama.ama-assn.org/cgi/content/full/300/15/1814)**Since EBM advocates that medical decisions should proceed from application of the current best evidence, an appreciation of how to evaluate, or grade, evidence is crucial to the application of best evidence in practice. *Conceptually, evidence starts simply with what is observed. Every individual observation is an isolated piece of evidence. To generate higher quality evidence, however, it is important to compile, organize, and evaluate those individual observations in a systematic way. Thus, while an anecdotal observation constitutes evidence regarding a single event, a more organized compilation of several observed events can constitute a case series, a higher level of evidence. An even more organized way to evaluate an event or an intervention is to use systematic observation, as in an uncontrolled or controlled trial. A meta-analysis provides even higher quality evidence by systematically grouping together and synthesizing the results of multiple trials. Thus, the more systematic an approach that is taken to gathering and organizing evidence, ranging from the individual anecdote up to the meta-analysis of controlled trials, the higher quality the evidence.

    *With the growing appreciation for and application of EBM, efforts have also been made to provide appropriate guidance for clinicians seeking to understand the quality of evidence behind published recommendations and guidelines. The United States Preventive Services Task Force provided one of the first classifications of evidence, with the 1989 publication of the original Guide to Clinical Preventive Services. (A) The 5 level system presented to grade USPSTF recommendations was designed to reflect the quality of evidence underlying each of the preventive interventions discussed. (B) Subsequent years have seen a proliferation of evidence-rating schemes and classifications, all seeking to provide some guide to assessing and ranking the quality of published medical evidence. Evidence rating systems have been published by the Centre for Evidence-Base medicine in Oxford, UK, the American College of Cardiology, the American Academy of Family Physicians, and many other groups. While some early evidence-rating schemes erred on the side of too much detail and too many categories and sub-categories, more recent rating systems have focused on practicality and ease of understanding in clinical practice. The most recent comprehensive approach to providing a unified evidence rating system is the GRADE approach, (C) under development by the GRADE working group. (D)

    (A) U.S. Preventive Services Task Force. Guide to clinical preventive services: an assessment of the effectiveness of 169 interventions. Baltimore: Williams & Wilkins, 1989.

    (B) Woolf SH and Atkins D. The evolving role of prevention in health care: Contributions of the U.S. Preventive Services Task Force. Am J Prev Med 2001;20(3S):13-20 .

    (C) Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schnemann HJ; GRADE Working Group. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008 Apr 26;336(7650):924-6. PMID: 18436948 (http://www.bmj.com/cgi/content/full/336/7650/924)

    (D) http://www.gradeworkinggroup.org/*The United States Preventive Services Task Force (USPSTF) provides 4 grades of recommendations, depending on the certainty that an intervention or preventive measure is more likely to cause harm or benefit. USPSTF also gives an I rating to interventions for which it is felt the evidence is insufficient to balance risk vs benefit. (A)

    (A) USPSTF Grade Definitions. (available at: http://www.ahrq.gov/clinic/uspstf/grades.htm)*The GRADE system for rating evidence is currently under development by the GRADE working group. (A) This system is designed primarily to be used by authors of guidelines, and software is also being developed to guide use of this system in evidence appraisal, but an overview of the principles underlying GRADE is illustrative of the issues that need to be considered in evidence rating. (B)

    http://www.gradeworkinggroup.org/

    Guyatt GH, Oxman AD, Kunz R, Falck-Ytter Y, Vist GE, Liberati A, Schnemann HJ; GRADE Working Group. Going from evidence to recommendations. BMJ. 2008 May 10;336(7652):1049-51. PMID: 18467413

    *While evidence-classification and rating may seem a daunting challenge, the basic premise is that the more systematic the observations that are available (eg, RCT instead of just a case series) the better the quality of evidence. And, since EBM seeks to apply the current best evidence it is important to see for the highest quality studies that are available to address a given clinical question.

    ***In order to decide what to do in practice, we also need to know how well know when we are there that is, what kind of outcomes do we seek? Medical outcomes can be broadly grouped into 3 categories. Some outcomes (eg, blood pressure or cholesterol levels) are merely surrogate markers of disease. We measure these surrogate markers because we think they tell us something prognostically about the expected course of a persons disease process, but they do not directly impact how a patient feels from day to day. Others measure actual stage or extent of disease (eg, the stage of a diabetic ulcer, or the angiographic extent of disease). These may have a more direct bearing on a patients quality of life or extent of suffering, but still do not provide direct measures of long-term quality of life. The most important outcomes for guiding medical decisions are those that affect how patients feel and the quality of their lives that is, patient-oriented outcomes such as mobilty, suffering, longevity, and other considerations that bear directly on how a patient experiences his or her quality of life. In short, patient oriented outcomes have primarily to do with long-term morbidity or mortality. (A)

    (A) Slawson DC, Shaughnessy AF, Ebell MH, Barry HC. Mastering medical information and the role of POEMs--Patient-Oriented Evidence that Matters. J Fam Pract. 1997 Sep;45(3):195-6. PMID: 9312554An even simpler way to break down the types of outcomes that may be considered is into disease oriented outcomes such as blood sugar, blood pressure, flow rate, coronary plaque thickness, or patient oriented outcomes such as reduced morbidity, reduced mortality, symptom improvement, improved quality of life, or lower cost. (A)

    (A) Ebell MH, Siwek J, Weiss BD, Woolf SH, Susman J, Ewigman B, Bowman M. Strength of recommendation taxonomy (SORT): a patient-centered approach to grading evidence in the medical literature. Am Fam Physician. 2004 Feb 1;69(3):548-56. PMID: 14971837 (http://www.aafp.org/afp/20040201/548.html)In applying evidence regarding patient-oriented outcomes, it is also important to work collaboratively with the patient to determine which outcomes are most important to the patient, as this will guide the decision as to which interventions to pursue. Three examples illustrate this point: 1) Topical antibiotics for bacterial conjunctivitis may improve early and late resolution rates, but nearly all cases ultimately have complete remission. (A) 2) Digoxin for symptomatic heart failure provides no significant difference in mortality but is associated with lower rates of hospitalization and of clinical deterioration. (B) 3) Long-acting beta-2 agonists for asthma are effective in reducing symptoms but may increase mortality or exacerbations. (C) Thus, it is vital to discuss the desired target outcome with a patient when deciding how to apply evidence to a specific patients situation.

    Sheikh A, Hurwitz B. Topical antibiotics for acute bacterial conjunctivitis: Cochrane systematic review and meta-analysis update. Br J Gen Pract. 2005 Dec;55(521):962-4. PMID: 16378567Hood WB Jr, Dans AL, Guyatt GH, Jaeschke R, McMurray JJ. Digitalis for treatment of congestive heart failure in patients in sinus rhythm: a systematic review and meta-analysis. J Card Fail. 2004 Apr;10(2):155-64. PMID: 15101028Cates CJ, Cates MJ. Regular treatment with salmeterol for chronic asthma: serious adverse events. Cochrane Database of Systematic Reviews 2008, Issue 3. Art. No.: CD006363. DOI: 10.1002/14651858.CD006363.pub2.*The Strength of Recommendation Taxonomy is one system of evidence grading, developed by a collaboration among family medicine editors, that seeks to provide a user-friendly approach to classifying evidence in terms of both evidence quality, and the degree to which it bears on patient-oriented outcomes. (A)

    (A) Ebell MH, Siwek J, Weiss BD, Woolf SH, Susman J, Ewigman B, Bowman M. Strength of recommendation taxonomy (SORT): a patient-centered approach to grading evidence in the medical literature. Am Fam Physician. 2004 Feb 1;69(3):548-56. PMID: 14971837 (http://www.aafp.org/afp/20040201/548.html)*Clinicians striving to practice EBM, applying the best current evidence to patient care, will still be faced (sometimes frequently) with the task of reconciling competing or conflicting recommendations. Oxman, Glasziou and Williams provide some helpful suggestions for issues to consider when trying to make clinical sense out of seemingly conflicting evidence or recommenations. Questions to consider regarding the evidence presented include: Were the clinical questions different? Were different studies considered? Were the results analyzed differently? Was the quality of evidence assessed differently? Questions of outcome or consequences that need to be considered include: Did the effect estimates for important outcomes differ? Did judgments about evidence quality differ? Were health consequences weighed differently? Were economic consequences considered differently?

    (A) Oxman AD, Glasziou P, Williams JW Jr. What should clinicians do when faced with conflicting recommendations? BMJ. 2008 Nov 28;337:a2530. doi: 10.1136/bmj.a2530. PMID: 19042938

    **Increasing emphasis on evidence-based medicine has also brought to the fore the potential tension between the presumed autonomy of the individual practitioner or physician, and the recognized value of informed decision making on the part of patients. Evidence and information systems can help educate both patients and physicians on the potential benefits and risks of different courses of action, but engaging patients in this process can seem threatening to physicians who are used to being autonomous professionals given full charge for the patients care. A recent JAMA editorial highlighted some key principles for addressing this tension:Physicians must recognize the role of being the patients agent in helping make informed decisions to maximize benefit at reasonable costMedical students and residents should be educated to approach care as the patients agent in making informed decisions, rather than solely as an autonomous decision makerThe evidence for and approach to developing standards should be standardized.

    (A) Mathews SC, Pronovost PJ. Physician autonomy and informed decision making: finding the balance for patient safety and quality. JAMA. 2008 Dec 24;300(24):2913-5. PMID: 19109120 In summary:Evidence based medicine is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. Evidence is essentially about observation, but high-quality evidence requires systematic observationEvidence is best applied in the pursuit of patient-oriented outcomes

    ***